Move GumbelSoftmax OP to phi #39873
Merged
Move GumbelSoftmax OP to phi #39873
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
YuanRisheng
reviewed
Feb 24, 2022
paddle/phi/infermeta/unary.cc
Outdated
| PADDLE_ENFORCE_GE( | ||
| axis, | ||
| -rank, | ||
| paddle::platform::errors::InvalidArgument( |
paddle/phi/infermeta/unary.cc
Outdated
| PADDLE_ENFORCE_LT( | ||
| axis, | ||
| rank, | ||
| paddle::platform::errors::InvalidArgument( |
paddle/phi/infermeta/unary.cc
Outdated
| PADDLE_ENFORCE_EQ( | ||
| out.dims(), | ||
| dout.dims(), | ||
| paddle::platform::errors::InvalidArgument( |
|
|
||
| PADDLE_ENFORCE_GT(temperature, | ||
| 0, | ||
| paddle::platform::errors::InvalidArgument( |
| default: | ||
| PADDLE_ENFORCE_LE(out->dims().size(), | ||
| 6, | ||
| paddle::platform::errors::InvalidArgument( |
zyfncg
reviewed
Feb 24, 2022
paddle/phi/infermeta/unary.h
Outdated
| int axis, | ||
| MetaTensor* out); | ||
|
|
||
| void GumbelSoftmaxGradInferMeta(const MetaTensor& out, |
There was a problem hiding this comment.
反向的InferMeta函数放到infermeta目录下的backward.h/.cc文件里
chenwhql
reviewed
Feb 24, 2022
paddle/phi/infermeta/unary.h
Outdated
| int axis, | ||
| MetaTensor* out); | ||
|
|
||
| void GumbelSoftmaxGradInferMeta(const MetaTensor& out, |
There was a problem hiding this comment.
grad infermeta统一放到backward.h/cc中
| // generate uniform random number | ||
| const int size = size_to_axis * size_from_axis; | ||
| std::uniform_real_distribution<T> dist(0.00001, 1); | ||
| auto engine = paddle::framework::GetCPURandomEngine(0); |
There was a problem hiding this comment.
这一点和秋良同步下，秋良后续应该会把generator也移过来
There was a problem hiding this comment.
重新修改了下，现在这个OP里使用的engine也同步换过来了，distribution仍然保持使用std的
|
|
||
| namespace phi { | ||
|
|
||
| static inline int CanonicalAxis(const int axis, const int rank) { |
| const int size_to_axis = SizeToAxis(axis, dx->dims()); | ||
| const int size_from_axis = SizeFromAxis(axis, dx->dims()); | ||
| DenseTensor dx_2d, out_2d, dout_2d; | ||
| dx_2d.ShareDataWith(*dx).Resize({size_to_axis, size_from_axis}); |
There was a problem hiding this comment.
这个ShareDataWith如果想避免CI检查的话，可以直接改成copy构造，DenseTensor dx_2d(*dx)，目前实现应该是一样的
|
当前迁移继续推进即可，但我后面倾向于将softmax几个kernel实现都放到softmax_kernel.h中，减少一些文件 @YuanRisheng |
|
errors命名空间问题和反向Infermeta放置位置已更改 @YuanRisheng @MingMingShangTian @zyfncg @chenwhql |
zyfncg
approved these changes
Feb 25, 2022
chenwhql
approved these changes
Feb 26, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Function optimization
PR changes
OPs
Describe
Move GumbelSoftmax OP to phi.